feat: User Selectable GPUs and GPU-based Model Recommendations #1730
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hiento09
added
P1: important
hiento09
force-pushed
the
feature/user-select-gpus
branch
from
d02c1dc to
22a24d2
Compare
hiento09
changed the title
hiento09
force-pushed
the
feature/user-select-gpus
branch
from
574b157 to
0bb5315
Compare
|
|
|
hi @hiento09, from the screenshot, looks like we are missing the separation line between the GPU and Experimental option, can you double check? On the other hand, I would recommend shortening the warning message if possible 🤔 e.g. Enabling abc, xyz..., may .... Many thanks |
hiento09
changed the title
This was referenced Jan 31, 2024
|
I included the design in this ticket for this implementation: DesignWhen users have compatible GPU(s)
When click on choosing GPU, there are 2 cases:
When users select more than 1 GPU
When users turn off GPU acceleration although they have compatible GPUShow warning icon -> when hover, users see tooltip suggestion that users should turn it on: 
When users don't have any compatible GPU
|
louis-menlo
force-pushed
the
feature/user-select-gpus
branch
from
fce3fe7 to
27a703e
Compare
hiento09
force-pushed
the
feature/user-select-gpus
branch
from
27a703e to
1eac907
Compare
hiro-v
force-pushed
the
feature/user-select-gpus
branch
from
1eac907 to
b4dc2ce
Compare
louis-menlo
approved these changes
Feb 6, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
Reason for changes
Fixes Issues
Images
By default, the most powerful GPU will be selected if the user enables GPU mode for the first time. After that, the system will not intervene, allowing the user to choose the GPU they want to use. The next time the app is launched, it will load the GPU according to the user's configuration.
When GPU mode is enabled, a model must certainly be selected. The user will not be able to deselect all GPUs
When the user chooses to use more than one GPU, a warning will be displayed to remind the user about multi-GPUs mode.
The bottom bar monitoring GPU resources will also display the corresponding selected GPUs.
The current Hub will also display suggested models depending on whether it's in RAM mode or GPU mode. For GPU mode, it will base its suggestions on the total VRAM of the selected GPUs
This is a suggestion if I enable all three GPUs. Notice the bottom bar below is enabling three GPUs, so I can load a heavy model up to 18GB as suggested here.
And here, when I try to disable one GPU, you will see that the Hub will recommend differently.
Example settings.json file
{"notify":true,"run_mode":"gpu","nvidia_driver":{"exist":true,"version":"531.18"},"cuda":{"exist":true,"version":"12"},"gpus":[{"id":"0","vram":"12282","name":"NVIDIA GeForce RTX 4070 Ti\r"},{"id":"1","vram":"6144","name":"NVIDIA GeForce GTX 1660 Ti\r"},{"id":"2","vram":"6144","name":"NVIDIA GeForce GTX 1660 Ti"}],"gpu_highest_vram":"0","gpus_in_use":["0","1"],"is_initial":false}